Method of Providing Camera Views About an Object or Area

ABSTRACT

A method of providing an omnidirectional view about an object by providing at least two camera modules, each module having a plurality of cameras disposed about the periphery of each module with each camera having a predetermined field of view. The modules are directly attached to a respective predetermined position on a surface of the object to provide up to a 360 degree view about the object when the fields of view of each camera are combined from each module. Electrical communication is enabled between the plurality of cameras and at least one position external to the modules to enable viewing about the object from the external position.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is related to U.S. patent application entitled“Miniature Camera Module”, attorney docket No. 10039 and U.S. patentapplication entitled “System for Providing Camera Views”, attorneydocket No. 10059 both of which are assigned to the same assignee as theassignee herein, filed concurrently herewith and hereby incorporated byreference.

BACKGROUND OF THE INVENTION

The present invention relates generally to camera modules and moreparticularly to a method of utilizing one or more low-profile,inexpensive, miniature camera modules to provide omnidirectional,panoramic, panospheric and/or hemispherical views about an object orarea.

SUMMARY OF THE INVENTION

A method of providing an omnidirectional view about an object byproviding at least two camera modules, each module having a plurality ofcameras disposed about the periphery of each module with each camerahaving a predetermined field of view. The modules are directly attachedto a respective predetermined position on a surface of the object toprovide up to a 360 degree view about the object when the fields of viewof each camera are combined from each module. Electrical communicationis enabled between the plurality of cameras and at least one positionexternal to the modules to enable viewing about the object from theexternal position.

DESCRIPTION OF THE DRAWINGS

The following detailed description will be better understood when readin conjunction with the appended drawings, in which there is shown oneor more of the multiple embodiments of the present disclosure. It shouldbe understood, however, that the various embodiments of the presentdisclosure are not limited to the precise arrangements andinstrumentalities shown in the drawings.

FIG. 1. is a perspective view illustrating one embodiment of a cameramodule that can be utilized in the present invention;

FIG. 2. is a front plan view illustrating the camera module of FIG. 1attached to an unmanned vehicle, such as a robot;

FIG. 3. is a perspective view illustrating one or more of the cameramodules of FIG. 1 attached to another type of unmanned vehicle;

FIG. 4 is a is a perspective view of a head mounted display illustratingmultiple camera modules of FIG. 1 secured thereto;

FIG. 5. is a partial top view of the unmanned vehicle of FIG. 3illustrating one embodiment of the potential field of view of the cameramodules; and

FIG. 6 is a perspective view illustrating multiple camera modules ofFIG. 1 utilized with an aerial vehicle such as an unmanned aerialvehicle or drone.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

The present invention will be described in detail with reference toembodiments that represent examples of the present invention and are notintended to limit the scope of the invention. Although specific elementsand configurations are described to provide an understanding of theinvention, it is to be understood that the specific embodiments,elements and configurations provided are for illustrative purposes only.Other configurations will be recognized by those of ordinary skill inthe art without departing from the teachings of the present invention orthe scope of the appended claims.

Referring to FIG. 1, one embodiment of a camera module that may be usedin the present invention is generally illustrated by reference numeral10. The module 10 is illustrated having a generally hemispherical shape,although the size and shape may vary, and may be a miniature module orthe like as is illustrated in FIG. 1 with respect to a United Statesquarter dollar 12.

Although not illustrated, the module 10 may also be in the form of ahemispheroid, either oblate (earth shaped) or prolate (rugby footballshaped), or in the form of a plurality of substantially flat or obliquesurfaces joined together to form similar types of shapes. Although themodule 10 preferably is formed as an oblique object, it is to beunderstood that the module 10 can have a variety of shapes and sizeswithout departing from the teachings of the present invention or theappended claims.

The unique shape and size of the module 10 itself enables a number ofnovel features to be realized by the present invention. These features,among others, include overlapping fields of view for multipleredundancy, a direct camera view for virtually every direction,unobtrusive and inconspicuous operation to resist or eliminate beingblocked, detected and/or disabled, and substantially oblique surfacesthat, combined with the material of the module 10, can resistdisablement from a foreign object or the like as will be described inmore detail herein.

The module 10 substantially includes a hemispherical housing 14 having aplurality of apertures 16 therethrough and a substantially flat surface18 for mounting of the module 10 against a desired article (generallyillustrated in FIGS. 2 and 3). The module 10 also includes a pluralityof cameras 20, each camera 20 being preferably secured within theconfines of the housing 14 so that a lens 22 of each camera 20 providesa desired field of view through a respective aperture 16. Although themodule 10 is illustrated with seven cameras 20, the number of cameras 20and their position with respect to the housing 14, the fields of viewand the lenses provided, among other features, may vary.

The housing 14 is designed as a substantially hollow, hemisphericalmember with a desired thickness and can be formed from any desiredmaterial such as plastic or metal (including steel or titanium, forexample) or the like as well as a variety of composite materials,including any type of bullet resistant or bullet proof materials such asKevlar, Lexan or the like so long as the module 10 functions as desired.The housing 14 may also be used in outdoor environments by making itwater resistant or waterproof by making it completely sealed against theenvironment and may be protected by any type of desired armor. It is tobe understood, however, that the material of the housing 14, or anyother elements or features of the housing 14, may vary without departingfrom the teachings of the present invention or the appended claims.

If desired, the module 10 may be designed to be completelyself-contained with all electronics included inside the housing 14 suchas microprocessor(s), data storage, memory, power and externalcommunications such as by any wireless method including Bluetooth, forexample. With such a design, all processing of the videos from thevarious cameras 20 can be accomplished by the module 10 itself,including stitching together multiple or all of the various views of thecameras 20 into one data stream to be stored within, or transmitted tothe exterior of, the module 10.

Alternatively, individual views of the cameras 20 can be provided inseparate video streams for internal storage or external transmission.With such a design, individual camera views can be selected for viewingand two or more selected views can be stitched for composite viewing.

Although the flat surface 18 of the housing 14 is not illustrated indetail, the flat surface 18 may allow for access to the interior of thehousing 14 for mounting or replacement of the cameras 20 therein such asby the use of an openable or removable plate, door or the like or anyother type of access. The housing 14 may also include a mountingstructure or element (not illustrated) either as part of the flatsurface 18 or any other part of the housing 14 including, but notlimited to, such mounting methods as screws, brackets, clips, adhesives,and hook-and-loop fasteners, for example.

The apertures 16 of the housing 14 may be through-apertures having thesame diameter through the housing 14, may be tapered or take on anyother desired size, shape or configuration. Alternatively, the apertures16 may be replaced with pockets (not illustrated) formed in the housing14 to accept the cameras 20 therein.

FIG. 2 illustrates one embodiment of the module 10 mounted to anunmanned vehicle such as a robot 24 or the like where the module 10 andcameras 20 can be used in a “visual odometry” type of application wherethe position, operation and/or orientation of the robot 24 is determinedby analyzing images from one or more cameras 20. If desired, multiplemodules 10 (not illustrated) may also be used with the robot 24 toprovide multiple views from a variety of positions about the robot 24.

As FIG. 3 illustrates, one or more modules 10 may also be mounted to anunmanned transport vehicle 26 intended for transporting items or thelike. When the transport vehicle 26 is in the form of a substantiallyrectangular object as illustrated, the views of the modules 10 enablemultiple overlap between modules 10 as described below with regard toFIG. 5.

As FIG. 4 illustrates, one or more modules 10 may also be used inconjunction with a head mounted display 28 illustrated in the form of ahelmet. It is to be understood, however, that the number and position ofmodules 10 utilized at any time can vary and the module 10 may beutilized in a variety of applications other than those illustrated.

The use of a module 10 along with multiple cameras 20 and/or multiplemodules 10 and the design and shape of the modules 10 themselvesrepresents a dramatic shift away from current remote camera design andapplication. For example, by providing multiple cameras 20 in a module10 arranged according to the teachings of the present invention a newlevel of redundancy in camera views is achieved in the event of amalfunction of one or more of the cameras 20 as will be describedfurther below. Intentional or unintentional disablement of the module 10can also be more difficult since, due to the overlapping views, eachcamera 20 in each module 10 must be disabled to prevent a desired viewand access to each camera 20 or module 10 may be difficult or too timeconsuming, for example.

By using multiple modules 10, an additional level of redundancy incamera views can be provided according to the teachings of the presentinvention by providing multiple sets of cameras 20 in multiple, separatelocations selected to provide a heightened level of view redundancy oroverlap. This not only can be beneficial in the event of the malfunctionor disablement of one or more cameras 20, but in the malfunction ordisablement of one or more modules 10.

With reference to FIG. 5, when multiple modules 10 are used on arectangular object such as the transport vehicle 26 significant portionsabout the transport vehicle 26 are covered by two modules 10. In thisembodiment, coverage by two modules 10 is illustrated by the four singlelined sections 48 while coverage by just one module 10 is illustrated bythe four unlined sections 50. Of course, for each module 10 sevencameras 20 provide the desired hemispherical view with overlappingfields of view as described above.

With multiple modules 10, intentional or unintentional disablement ofone or more modules 10 is more difficult or unlikely. This isparticularly important when the modules 10 are used in a securitysurveillance setting or in conjunction with a robot or other type ofunmanned vehicle or craft and even more so in a combat environment.Specifically, when used in a robot or the like, when the camera viewingsystem is inoperable or disabled the robot is unable to function and ifremotely controlled the user is unable to continue to “see” where therobot is and/or needs to be moved.

When used in a hostile environment, such as a surveillance setting orcombat zone, an entity who may be interested in disabling the robot canmerely take measures to attempt to disable a the camera 20, such as witha projectile, cover, spray paint, implement or other member. Withmultiple modules 10, while an entity may disable one or more modules 10the operator may still be able to utilize an existing module 10 notdiscovered or disabled by an entity to move the robot out of the area.

The design of the module 10 itself also adds to the ability of anoperator to maintain control over a robot. The small size of the module10 provides an extremely low profile making the module 10 unobtrusiveand relatively inconspicuous compared to existing modules. If desired,the module 10 can also be camouflaged to blend in with a particularbackground as illustrated in FIG. 1 to make it even more difficult tospot and potentially disable.

The hemispherical, hemispheroid or other oblique design of the module 10aids in preventing disabling the module 10 by providing oblique surfacesthat may deflect a projectile or implement. This shape also makesbreaking off of the module 10 from a surface it is attached to moredifficult, particularly if some type of security resistant mounting isemployed. When the oblique surfaces of the module 10 are combined withthe module 10 being formed from a strong material, such as metal or abullet resistant or bullet proof material or the like, disabling of themodule 10 from a projectile or implement is less likely.

As readily can be understood from FIG. 1, due to the positioning of thecameras 20 about the module 10, when the video of each camera 20 iscombined a hemispherical, panoramic or “panospheric” view (hereinafter“view” or “hemispherical view”) about the module 10 is provided. Theview may extend up to at least 180 degrees in any direction about theperiphery of the module 10 but may vary depending on the types ofcameras 20 and lenses 22 utilized and the positions about the housing14.

When multiple modules 10 are used, such as shown with the head mounteddisplay 28 of FIG. 4, for example, the view about the periphery of thedisplay 28 extends well beyond 180 degrees. Although four modules 10 areillustrated, any number of modules 10 can be positioned on the robot 24or transport vehicle 26 on any side thereof including the top and bottomso that a full 360 degree view is provided about the robot 24 ortransport vehicle 26.

The cameras 20 can be any type of camera, video or still, so long asthey function as desired, including, but not limited to cameras having amicro lens, miniature and subminiature cameras, fiber optic cameras, CCDcameras, active pixel or CMOS image sensors, or the like includingfuture camera types and styles. Additionally, various software requiredto “stitch” the various views together is readily available or can beindependently developed to fit the particular application.

Preferably, the cameras 20 are not high level resolution cameras,rather, lower resolution cameras 20 are utilized the resolution of whichdepends on the particular application. In many applications, the qualityof the image being captured is not necessarily an important criteria,rather, capturing a desired image with an acceptable level of quality isall that is necessary.

The use of a plurality of lower resolution cameras 20 represents animportant shift in the art that enables the desired views in a moreeconomical package enable applications that previously were costprohibitive. The cost of modules 10 with lower resolution cameras 20 canbe low enough to enable modules 10 to be disposable if desired.

A variety of electronics and power supply (not illustrated) may be usedalong with the cameras 20 to provide the desired view. The electronicsand power supply may be contained within the confines of the housing 14or outside the housing 14 such as within a portion of the robot 24,transport vehicle 26, or head mounted display 28 or in a separate unitutilized in conjunction with the robot 24, transport vehicle 26 or headmounted display 28. The electronics may be hard wired to the cameras 20and other peripheral devices or be connected wirelessly, if desired.

The module 10 preferably is designed as a self-contained readilyreplaceable unit to assist in initial mounting and subsequent repair bymerely replacing the module 10. In this respect, the module 10 can beused substantially as an interchangeable platform for use with multiplevehicles or other objects as is more frequently being requested inmilitary or other operations to reduce costs incurred from the use ofmultiple platforms.

Additionally, a minimal amount of electronics may be included with thecameras 20 or within the housing 20 for wireless communication with avariety of external peripheral devices. Such devices may include, forexample, remote displays, microprocessor controlled devices, varioususer input/output devices or the like. External viewing can be acomposite view of all of the cameras 20, the view of any individualcamera 20 or any combination of two or more video cameras 20 if desired.

In one preferred embodiment, the view provided by the cameras 20 may befed to an operator or other individual or individuals at a locationproximate to or remote from the module(s) 10. This view can, forexample, be used to manipulate the robot 24 or transport vehicle 26about various terrain and environments or to accomplish various tasks.It is to be understood that to manipulate the robot 24 or transportvehicle 26 discrete electronic and power systems separate from theelectronics of the module 10 may be employed.

Additionally, the head mounted display 28 of FIG. 4 may be used by awearer to provide a full view about the viewer to both the viewer aswell as one or more individuals at remote locations. Although the headmounted display 28 is illustrated with four modules 10 positioned on thefront, rear and two sides respectively, it is to be understood that thenumber and position of the modules 10 can vary.

One of the many important features of the module 10 is the ability tosimultaneously capture and display all data with regard to all views ofeach camera 20 in real time or substantially real time. This enablesviewers at remote locations to choose one or more cameras 20 to selectthe view or composite view they are interested in viewing regardless ofthe direction the wearer is facing. This may enable the remote viewersto draw the attention of the wearer to another direction that may be ofinterest to them or to watch one or more directions for the safety ofthe wearer, such as in a military, police or similar applications.

Additionally, since various camera views from multiple cameras 20 aresimultaneously captured or “pre-loaded”, delays in changing views withthe head mounted display 28 upon turning of the wearer's head or turningof the robot 24 or transport vehicle 26 is reduced or eliminated. Whenusing a single camera rather than multiple cameras 20 as in the presentinvention there will be a lag or delay in views when the camera is movedwhich is undesirable and perhaps dangerous in certain applications.

Similarly, when multiple modules 10 are utilized providing various viewsfrom multiple cameras 20 from each module 10, the various camera viewsfrom multiple cameras 20 are simultaneously captured or “pre-loaded” toeliminate delays in views. This is an important feature for not only thehead mounted display 28 but also the robot 24 and transport vehicle 26.

For example, with a head mounted display 28 if only one module 10 isused the views provided by the multiple cameras 20 is more of apanoramic or “panospheric” view that extends in a hemisphericaldirection substantially in the direction the module is facing. As thewearer turns his/her head say to the left, the view that previously wason the left periphery of the wearer now becomes the front view withrespect to the wearer. At the same time, a new left periphery view isprovided to the wearer or remote viewer.

When using multiple modules 10, the wearer is always provided with aclear view of the direction he/she is facing. It also enables a wearerto more quickly move his/her head back and forth without losing the viewof the direction he/she is facing.

Another important feature of the module 10 is the ability to enable moreplacement positions with respect to an object, such as the robot 24 ortransport vehicle 26. This is not only due to the substantially smallsize of the module 10 but due to the use of multiple modules 10 thatbreak up the 360 degree view into several hemispherical views that mayoverlap each other to provide redundant or overlapping views. Typically,in existing applications if single or multiple cameras are provided in acentral location (not illustrated) for fixed position or rotation toprovide a wide view up to 360 degrees, that view readily can be blockedduring operation of the robot 24 or transport vehicle 26 by componentsof the robot 24 or transport vehicle 26.

The relatively small sizes capable of the module 10 enable moreplacement positions on an object to provide increased number of views orplacement where a camera system typically may not fit. Additionally, dueto the small size of the module 10 it can be placed closer to an objector surface to reduce the size of any potential blind spot that may occurnear the object or surface itself.

When multiple modules 10 are used with a robot 24 or other unmannedtransport vehicle 26 since the robot 24 or vehicle 26 can typically movein multiple directions, when one module 10 is broken or disabled therobot 24 or vehicle 26 can still function. In this situation, anoperator can merely switch views to another module or modules 10 andmerely direct the robot 24 or vehicle 26 to move in another directionwhile the operator can still manipulate the robot 24 or vehicle 26 asdesired.

As discussed above, providing a plurality of modules 10 each withmultiple miniature inexpensive cameras 20 in a particular applicationrepresents a complete divergence from the current state of cameramodules where single, large, expensive modules typically are employed.Such single modules can interfere with other objects, can readily bespotted and disabled and provide no redundancy in the event of failureor obstructions, among other drawbacks.

These drawbacks are unacceptable for many applications of cameras whichhas severely limited their application in many areas that could benefitfrom their use, such as military, robotics and surveillance among otherareas. For example, FIG. 6 illustrates one potential militaryapplication where an unmanned aerial vehicle or UAV 56 can be fittedwith any number of modules 10 of the present invention.

As can readily be appreciated, the modules 10 do not interfere with anyof the other mechanisms of the UAV 56 such as the propulsion mechanisms,ordinance, payloads or even aerodynamics. Typical existing modules tendto be single modules mounted on a pedestal or the like and the viewscannot be obstructed or a remote operator cannot see all angles andaccurately control the UAV 54 as necessary. This not only is a majorconcern with existing modules for applications to UAVs but for anyapplication such as a robot or other vehicle such as a car, truck or thelike.

Additionally, it has become increasingly popular to provide full 360degree panoramic views for use in movies, simulators, attractions suchas amusement parks or the like, video games, internet cites, cell phonesand GPS units or any other type of application. In these applications ittypically is desirable to provide actual footage of an existingenvironment such as a neighborhood, city, golf course, race track,flight path or any other similar application.

For example, flight training for pilots typically is accomplished in aflight simulator (not illustrated). During such training, in order toprovide the trainee with the most realistic training possible, thetrainee can be situated in an environment surrounded by multiple screenseach screen representing the views that may be encountered in actual useby looking out windows, monitoring video cameras or the like.

Typically these multiple views are computer generated as opposed toproviding actual footage of real life flying conditions such as terrain,cloud cover, other aircraft and the like. Providing those views asactual footage clearly is an advantage which can readily be provided inan economical manner with the modules 10 of the present invention.

Similarly, popular attractions at amusement parks provide a 360 degreevirtual reality exhibit taking customers across various parts of theearth. Use of the modules 10 of the present invention can help providesuch views in an economical manner as well as enable stitching of thebest possible views among redundant views for the best enjoyment of thecustomers.

In various internet, gaming, GPS and cell phone applications, forexample, views of actual locations on earth are increasingly popular.Such applications typically are provided by driving or walking aroundneighborhoods or other locations to capture desired views with one ormore cameras. Existing systems with one or more cameras typically arereadily blocked by various equipment or other items such as otherpeople, cars and similar impediments thereby preventing the unobstructedviews needed.

With the modules 10 of the present invention, multiple views of the samescene can readily be captured in a cost effective manner. This not onlycontributes to more applications being developed, but enables moreversatility in the views being selected for the benefit of the end userand eliminating unobstructed views in many if not all applications.Thus, a user can traverse a desired location once and rely on capturingthe desired footage thereby reducing the chance of having to retake thefootage again in the future due to obstructions or failure of one ormore modules or cameras.

The embodiments of the present disclosure may be implemented with anycombination of hardware and software. If implemented as acomputer-implemented apparatus, the embodiments of the presentdisclosure are implemented using means for performing all of the stepsand functions described above.

The embodiments of the present disclosure can be included in an articleof manufacture (e.g., one or more computer program products) having, forinstance, computer useable media. The media has embodied therein, forexample, computer readable program code means for providing andfacilitating the mechanisms of the embodiments of the presentdisclosure. The article of manufacture can be included as part of acomputer system or sold separately.

Although the description above contains many specific examples, theseshould not be construed as limiting the scope of the embodiments of thepresent disclosure but as merely providing illustrations of some of thepresently preferred embodiments of this disclosure. Thus, the scope ofthe embodiments of the disclosure should be determined by the appendedclaims and their legal equivalents, rather than by the examples given.

It will be appreciated by those skilled in the art that changes could bemade to the embodiments described above without departing from the broadinventive concept thereof. It is understood, therefore, that thisdisclosure is not limited to the particular embodiments disclosed, butit is intended to cover modifications within the spirit and scope of theembodiments of the present disclosure.

1. A method of providing an omnidirectional view about an object,comprising: providing at least two camera modules, each module having aplurality of cameras disposed about the periphery of each module witheach camera having a predetermined field of view; directly attachingeach module to a respective predetermined position on a surface of theobject to provide up to a 360 degree view about the object when thefields of view of each camera are combined from each module; andenabling electrical communication between the plurality of cameras andat least one position external to the module to enable viewing about theobject from the external position.
 2. The method as defined in claim 1,wherein the electrical communication is a wireless type of electricalcommunication.
 3. The method as defined in claim 1, wherein the objectis at least one of a robot or other type of unmanned vehicle or craft.4. The method as defined in claim 1, wherein the external position isexternal to both the module and the object.
 5. The method as defined inclaim 1, wherein the views provided by each camera may be combined toprovide a segmented view about the object.
 6. The method as defined inclaim 1, wherein the 360 degree view is a combination of at least twosubstantially hemispherical views.
 7. The method as defined in claim 1,wherein each module is provided substantially in the form of ahemisphere and the flat portion of the hemisphere is attached to theobject.
 8. The method as defined in claim 1, wherein viewing about theobject can take place in more than one external position.
 9. The methodas defined in claim 1, wherein each module has a low profile and a flatsurface for directly attaching to the surface of the object.
 10. Themethod as defined in claim 1, wherein the modules are positioned so thatthey do not interfere with any movement or operation that may beperformed by the object.
 11. The method as defined in claim 10, whereinthe object is at least one of a robot or other unmanned vehicle havingat least one mode of operation that may block at least a portion of atleast one camera.
 12. The method as defined in claim 1, wherein themodules may be used to provide views that enable the object to be movedor manipulated.
 13. The method as defined in claim 12, wherein when onemodule is inoperable, blocked or otherwise can not provide one or morecamera views the object can still be moved or manipulated by use of theat least one other module or camera.
 14. The method as defined in claim1, wherein each module is a completely self-contained module.
 15. Amethod of providing a real-time composite video image of an area,comprising: providing a platform; attaching at least two modules topredetermined portions of the platform, each module having a pluralityof video cameras disposed about the periphery of each module with eachcamera having a predetermined field of view; capturing video streamsfrom each camera; moving the platform as desired; transmitting the videostreams to a location remote from the platform; and stitching at leasttwo of the camera views together to provide a composite image as theplatform moves.
 16. The method as defined in claim 15, wherein stitchingis provided prior to the images being transmitted from the platform. 17.The method as defined in claim 15, wherein the platform is at least oneof a ground vehicle, manned aerial vehicle or an unmanned aerialvehicle.
 18. The method as defined in claim 15, including selecting at aremote location the individual or composite view desired.
 19. The methodas defined in claim 15, wherein the video streams can be transmitted tomore than one remote location.